Motion compensation and intraprediction are important aspects of video compression which predict the data in a current frame from previously transmitted frames or from previously encoded parts of the same frame. This allows transmission of only the residual differences between the current frame and a prediction.
Many video compression technologies apply such predictions including H.264, HEVC, Daala and many others. FIG. 1 illustrates an example of a generic video encoder 130 taking input data 110 and producing an encoded output 120, which may apply one or more of these technologies. An intra-prediction module 140 and motion compensation module 150 provide the encoder with predictions of samples from within the same picture or from reference pictures respectively.
However, the most optimal prediction might not yield the best compression efficiency due to additional overhead of syntax and syntax prediction. For example, when an optimal prediction is achieved by splitting the block and applying a large number of motion vectors, the syntax overhead for signalling this structure may be larger than the overhead had the block not been split and additional residual information added.
Furthermore, due to inherent properties of the human visual system, visual distortions in some areas of an image are more visible than in other areas. Therefore, an essential part in optimising a subjective quality of the resulting bit stream is the Rate Distortion Optimisation (‘RDO’) or mode decision process. This process selects, for a given bit rate, the optimal prediction (mode), motion vectors, and quantisation parameters, taking into account the introduced distortion and resulting bit rate. In many encoders the RDO uses objective metrics based upon mathematical distortions such as PSNR, SSE, and SAD. Consequently, any subjective distortion is not taken into account.
To improve the distribution of bits towards subjectively important areas, a pre-analysis process can be undertaken.